Richard Turner

Wednesday 22nd November 2017

Time: 4.00pm

Ground Floor Seminar Room

25 Howland Street, London, W1T 4JG

A Unifying Framework for (Deep) Gaussian Process Pseudo-Point Approximation

Gaussian processes (GPs) are flexible distributions over functions that enable high-level assumptions about unknown functions to be encoded in a parsimonious, flexible and general way. Although elegant, the application of GPs is limited by computational and analytical intractabilities that arise when data are sufficiently numerous or when employing non-Gaussian models. Consequently, a wealth of GP approximation schemes have been developed over the last 15 years to address these key limitations. Many of these schemes employ a small set of pseudo data points to summarise the actual data. In this talk I will develop a new pseudo-point approximation framework using Power Expectation Propagation (Power EP) that unifies a large number of these pseudo-point approximations. Unlike much of the previous venerable work in this area, the new framework is built on standard methods for approximate inference (variational free- energy, EP and Power EP methods) rather than employing approximations to the probabilistic generative model itself. In this way all of the approximation is performed at `inference time' rather than at `modelling time', resolving awkward philosophical and empirical questions that trouble previous approaches. Crucially, I show that the new framework includes new pseudo-point approximation methods that outperform current approaches. I will also show that they enable deep GPs to achieve state-of-the-art performance on a number of standard regression benchmark problems.